Superfluid-Insulator Transition and Fermion Pairing in Bose-Fermi Mixtures 
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It is well known that bosons on an optical lattice undergo a second-order superfluid-insulator 
transition (SIT) when the lattice potential increases. In this paper we study SIT when fermions 
coexist with the bosons. We find that the critical properties of particle-hole symmetric SIT with 
dynamical exponent z = 1 is modified when fermions are present; it either becomes a fluctuation- 
driven first order transition or a different second-order transition. On the other hand the more 
generic particle-hole asymmetric (with z = 2) SIT is stable against coupling with fermions. We also 
discuss pairing interaction between fermions mediated by quantum critical fluctuations near SIT, 
and show that the fermion pairing/superfluid transition temperature peaks near SIT when a single 
species of fermion is present. 



INTRODUCTION 

The observation of superfluid-insulator transition 
(SIT) of bosons on optical lattices [J, [2| is arguably 
one of the most important recent developments in the 
field of cold atom physics; it triggered a tremendous 
amount of interest and activities in studying strong cor- 
relation physics using optical lattices. More recently, 
much work has focused on strong interaction/correlation 
physics that involve fermions, like superfluidity resulting 
from fermion pairing, and realization of various exotic 
phases proposed in the context of electronic condensed 
matter physics Q. Another direction of thrust is Bose- 
Fermi mixtures on optical lattices; these are new many- 
body systems that have no direct analogy in electronic 
condensed matter systems. 

In this paper we consider boson SIT on optical 
lattices in Bose-Fermi mixtures in two- and three- 
dimensions, and consider two separate situations: (i) 
Bosonic atoms coexist with fermionic atoms, (ii) The 
bosons are molecules made of two fermionic atoms of 
different species (or opposite "spins"), and the numbers 
of fermions are different for different spins. Due to the 
imbalance some of the fermions are unpaired and coex- 
ist with bosonic molecules. While the physical settings 
are very different, we show that the effective field the- 
ory describing the SIT turn out to be the same for both 
cases. We find that the presence of fermion do not af- 
fect the critical properties of SIT for most parts of the 
phase boundary separating the superfiuid and insulat- 
ing phases, provided the Bose-Fermi interaction is suffi- 
ciently weak. On the other hand near some special point 
with (emerging) particle-hole symmetry, fermions do af- 
fect the critical properties of SIT. These are the subjects 
of section II below. 

Bosons mediate an attractive interaction between 
fermions, which leads to fermion pairing and superfluid- 
ity at low temperature. In section III wc discuss this pair- 
ing interaction near boson SIT, which is dominated by 
critical fluctuations. We show that under certain condi- 



tions the fermion superfluid transition temperature peaks 
near SIT. 

A few concluding remarks are offered in section IV. 



BOSON SUPERFLUID-INSULATOR 
TRANSITION 

Theoretically, the SIT is well understood in purely 
bosonic systems based on simple models like the Bose- 
Hubbard model[3,|i]: 

Hb = ~~tb ^ bjbj- flb'^nbi + ^'^nb^{nbi-l), (1) 

<ij> i i 

where bi is the boson annihilation operator on site i, 
nbi = blbi, U > is the on-site repulsion between the 
bosons, tb is the boson hopping matrix element and fib is 
boson chemical potential. The schematic phase diagram 
of ([T|) at zero temperature (T = 0) can be found in Fig. 
la, where there are two phases: At small tb/U the sys- 
tem is a Mott insulator with an integer number of bosons 
per site; for larger tb/U bosons dcclocalize and Bose con- 
dense, resulting in a superfluid phase. The superfluid- 
insulator transition (SIT) is continuous, and depending 
on the value of boson chemical potential fib there are two 
universality classes for the SIT[J|: (a) Generically, either 
particles or holes condense at the SIT, and the transition 
is the dilute Bose gas transition with dynamical expo- 
nent z = 2; (b) at the tips of the lobes of the Mott insu- 
lator phase, particles and holes condense simultaneously, 
and the SIT is in the universality class of d -I- 1 dimen- 
sional XY transition with z = 1 (here d is the spatial 
dimension of the system). The dynamical exponent z is 
the critical exponent that characterizes the relation be- 
tween energy/freqeuncy and momentum/wavevector of 
quantum critical excitations/fluctuations: ujk ^ ■ For 
the two universality classes of SIT here the values of z 
can be understood from the following considerations: (a) 
For the particle-hole asymmetric case, the critical exci- 
tations are either particles or holes that become gapless 
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FIG. 1: (Color on-line) (a) Schematic phase diagram of Bose 
Hubbard model. Solid lines are 2nd order phase boundaries 
separating superfluid phase from Mott insulator phases with 
fixed boson number ub per site. For each lobe of the Mott 
insulator characterized by a fixed ub, the superfluid-insulator 
transition (SIT) is triggered by the condensation of either 
particles (upper half of the lobe) or holes (lower half of the 
lobe), except at the tip of the lobe (blue dots) where particles 
and holes condense simultaneously, (b) In the presence of 
fermions, SIT may become 1st order near the tips of MI lobes 
(blue dotted lines). 



and start to condense at SIT; they have non-relativistic 
dispersion relation ujk — fc^/(2m*) (m* is their effective 
mass) , and z = 2 simply reflects this non-relativistic dis- 
persion, (b) In the particle-hole symmetric case, both 
particles and holes become gapless at SIT, and must be 
treated on equal-footing. Thus the effective field theory 
contains both particle and antiparticle (hole), which turn 
out to be a relativistic quantum field theory with emer- 
gent Lorentz symmetry. The Lorentz symmetry requires 
that space and time are symmetric, which guarantees 
that frequency must scale the same way as wavevector: 
LUk ^ k] this implies z = I. 

The presence of fermions introduces additional gap- 
less fermionic excitations, which can potentially change 
the critical properties of these two universality classes. 
We find that in the presence of fermions, the coupling 
between bosons and gapless fermionic excitations near 
the Fermi surface leads to singular terms in the effec- 
tive field theory for SIT. Such singular terms turn out 
to be irrelevant at the z = 2 fixed point, thus this fixed 
point is stable. On the other hand they lead to run- 
away renormalization group (RG) flows at the z = 1 
fixed point, thus destabilizing this fixed point; as a re- 
sult near the tips of the Mott insulator lobes SIT either 
becomes a fluctuation-driven first order transition, or be- 
comes a different 2nd order transition. In next section we 
will also study the pairing interaction between fermions 
mediated by quantum critical fluctuations near SIT, and 
show that under certain conditions the fermion pairing 
gap and transition temperature peaks near SIT. 

We begin our discussion with case (i) of Bose-Fermi 
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FIG. 2: Feynman diagrams representing terms in effective 
bosonic action generated from integrating out fermions, for 
case (i) of Bose-Fermi mixture. The solid and wavy lines are 
fermion and boson propagators respectively. 



mixture, described by the following Hamiltonian: 

H = Hg-tf {fj fj+h.C.)- fifJ2nf^ + VJ2^''^nf^, 

<ij> i i 

(2) 

where fi is the fermion operator on site i, Ufi = fjfi, and 
V is the Bose-Fermi interaction strength. Even though 
the Hamiltonian ^ is still quite simple, it actually has 
a very rich phase diagram^. To avoid complications as- 
sociated with instabilities like phase separation or bound 
state formation, we assume V to be weak and fermion 
density to be low. To derive the effective field theory for 
boson SIT, we use a functional integral representation 
of Eq. ([2]), and integrate out the (quadratic) fermionic 
degrees of freedom to obtain an effective action in terms 
of the bosons only. This procedure generates additional 
interaction terms for the bosons, in the form of 

X]^2n(gi,--- ,qn;q'i,--- ,qn)b{qi) ■ ■ -KqnMq'i) ■ ■ -biq'J. 

n 

(3) 

Here q ~ (q, lo) includes both momentum and frequency, 
and b is the bosonic field of the effective field theory. 
These terms are represented diagramatically in Fig. 2. 

Because there exist gapless excitations around the 
Fermi surface, tracing out fermions leads to terms in 
([3]) with singular coupling functions at long wave- 
length and/or low frequency (i.e., A2„(g) is non-analytic 
at small q and/or oj). The most studied cases with such 
singularities generated by tracing out gapless fermions 
are Hertz theories for magnetic transitions in metals Q, 
where due to the direct coupling between order param- 
eter and fermions, singularities appear at the quadratic 
level in the form of Landau damping. Such singular- 
ity not only changes the critical properties of the phase 
transition at Gaussian (or tree) level^Q], but also casts 
the validity of integrating out fermions in doubt 0. 

In the present case however, we find the singularities 
are less severe. This is because here the order parameter 
b does not couple to fermions directly; instead the cou- 
pling is in the form of density-density interaction between 
bosons and fermions. As a result no singularity appears 
in the quadratic action (i.e., no singularity in A2{q)); the 
bosons only receive a Hartree shift of chemical potential 
here. Consequently power counting and critical behavior 
do not change at this level. 
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Singularities do appear in higher order terms of ([3]), 
and the leading singularity is in a quartic term of the 
form: 



A4 / dqdcj — I 



^(q,^)l 



(4) 



where |6p(q, cj) is the Fourier transform of |&(r, r)|^.[8] 
The importance of this singular quartic term depends on 
the dynamical exponent z. For 2: = 2, w scales as |qp; 
it is clear that it is much less important than the regular 
quartic term J dr(ir|6(r, r)|^, and in fact scales to zero 
from a simple power counting analysis which leads to the 
RG flow for A4: 



rflogs 



(5 - d - 2z)A4 



(5) 



where s is the length rescaling factor that relates the 
length before and after an RG transformation: x' = x/ s 
(the corresponding transformation for time is t' = t/s^^ 
which provides the technical definition of dynamical ex- 
ponent z). Clearly A4 is an irrelevant coupling for z = 2 
and d > 1. We thus find that the z = 2 quantum critical 
point (QCP) of the SIT is stable against weak Bose-Fermi 
coupling in this case. 

The situation is very different at the particle- hole sym- 
metric QCP of SIT with z = 1. In this case Eq. ^ sug- 
gests A4 is marginal for d = 3 while relevant ior d — 2; it 
can thus change the critical properties of the SIT at this 
particle-hole symmetric QCP. Sachdev and MorinariQ 
analyzed a very similar model with the same type of sin- 
gular term in a different context. As they showed, the 
A4 coupling is actually non-renormalizable, as at each 
step of RG, new relevant or marginal couplings of dif- 
ferent types of singularities appear, as a result of which 
the standard RG procedure is unstable. As an alterna- 
tive they introduced a new field to decouple the singular 
coupling (H]), and corresponding coupling constants; the 
theory becomes renormalizable after such a modification. 
Within a perturbative RG treatment they find that the 
field theory yields no fixed point at weak coupling. 

There are two possible interpretations of the absence of 
fixed point within perturbative RG for the particle-hole 
symmetric case. It could indicate that the SIT is driven 
first order by fluctuations due to Bose-Fermi coupling 
around the tips of the Mott insulator lobes, as indicated 
in Fig. lb. Another more interesting possibility is that 
there actually exists a strong-coupling fixed point that is 
inaccessible to perturbative RG. If this turns out to be 
the case the SIT would remain 2nd order, but the critical 
properties would be controlled by the new fixed point at 
which the Bose-Fermi coupling strength flows to a uni- 
versal value; this would dramatically affect the fermion 
properties at the transition (to be discussed later). In 
this case the phase diagram remains that of Fig. la, 
but the critical behavior at the tips is no longer that of 



d+ 1 dimensional XY model. Unfortunately the tree-level 
RG equation ([5]) as well as the more elaborate treatment 
that includes loop contributions of Ref. cannot resolve 
which of the two scenarios is realized. This is because the 
perturbative RG flow equation ^ is well controlled only 
at weak coupling or small A4. While it clearly establishes 
the instability of the d + 1 dimensional XY fixed point 
(2: = 1) when the singular coupling A4 (due to presence of 
fermions) is present, one can not tell where the RG flow 
brings the system to, unless a more stable fixed point 
(with only one relevant perturbation) is found at weak 
coupling. On the other hand it would be very interest- 
ing to use numerical methods like quantum Monte Carlo 
to resolve which of the two scenario is realized; in such 
studies particle-hole symmetry can be ensured by fixing 
the boson number while varying £//th[loj. 

We now discuss case (ii) of Bose-Fermi mixture, in 
which the bosons are molecules made of two fermions 
with opposite spins, with density imbalance between 
them. Recently, much interest has focused on the prob- 
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lem of pairing in such imbalanced fermionic systems 
12, 13], with the hope of realizing novel superfluid phases 
different from the BCS state lj|. In the present paper we 
consider the relatively simple regime of strong pairing (or 
BEG) regime, in which one can think that all the minor- 
ity (-) fermions are paired with majority (-I-) fermions to 
form bosonic molecules, while the unpaired + fermions 
form a Fermi gas; phase separation does not occur when 
the imbalance is small in this regime[15]. As a result 
bosonic molecules coexist with unpaired fermionic atoms. 
The simplest model that contains the basic physics is the 
two-channel model on a lattice with one band for both 
the bosons and fermions: 



H = HB-tf J2 {fUaj+h.c.)~Y.^ii + ah)n,, 

- gY.^fljUh + h-^'-)- (6) 



Here cr = ±1 is the (pseudo)spin index for the fermions, 
fai is annihilation operator of the fermion on site i with 
spin a and n„i ~ flifai- The fermion species are at 
chemical potentials /i/o- = ji + crh, while the boson 
is at chemical potential fib — 2fj, — v. Here h is the 
"Zeeman field" which imbalances the fermion densities, 
and V is the "detuning" across the Feshbach resonance 
which scans between the BCS and BEC limits. As men- 
tioned earlier, we focus mainly in the regime of strong 
pairing or large negative detuning, \v\ ^ g,ii,t,U. In 
this regime fermions with opposite spins form closely 
bound molecules, and the system can sustain consider- 
able spin imbalance without phase separation 15| . The 
unpaired majority (-I-) fermions form a single Fermi 
surface 15|,[l6| with a volume satisfying a generalized Lut- 
tinger theorem 17| . It should be noted that while we have 
three types of particles, due to the g term in ^ there 
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FIG. 3: Same as Fig. 2, for case (ii) of Bose-Fermi mixture. 

are only two conserved charges: N„ ~ + flif<yi)- 

The boson superfluid and insulating phases here are de- 
fined with respect to the transport of conserved charge 
Physically this reflects the fact that at low energy 
minority (— ) fermions exist in the form of constituents 
of bosonic molecules at low-energy. 

We can again trace out the fermions in Eq. ([6]), which 
generates effective boson interaction terms of the form 
([3|) that are represented by a new set of diagrams of Fig. 
3. While the Hamiltonian ^ and the Feynman diagrams 
Fig. 3 look very different from those of the previous case, 
and in particular the superfluid order parameter (5) cou- 
ples directly to fermions here, the singularities generated 
by tracing out fermions turn out to be identical. Heuristi- 
cally it can be seen from the following considerations. In 
the Hamiltonian ([6]) the majority fermions are gap- 
pless but the minority (— ) fermions remain fully gapped] 
as a result the fermion Green's function has no pole at 
low-energy and can be treated as a number [ — \/\^ — h\). 
Diagramatically, one can thus shrink the — fermion lines 
to a point for the diagrams in Fig. 3, which reduces 
them to the same diagrams as those in Fig. 2. Alter- 
natively, one can also see this by first integrating out 
the fully gapped — fermions; this generates a quartic (or 
density-density) interaction between bosons and gapless 
-f fermions of the same form as in ([2]). They thus have 
the same singularities, and we conclude that these two 
cases of Bose-Fermi mixtures share the same phase dia- 
gram and critical properties of SIT. 



FERMION PAIRING NEAR BOSON 
SUPERFLUID-INSULATOR TRANSITION 

In this section we turn our discussion to the behavior of 
fermions, especially near SIT. It should be clear from the 
discussion above that the fermion physics is also similar 
for cases (i) and (ii), and we focus on case (i) below. It 
has been known for a long time[l3l that boson density 
fluctuations mediate an attractive interaction for case (i) 
of Bose-Fermi mixture, which at weak coupling takes the 
formlil,!!^ 



(7) 



where x(q, oj) is the boson density susceptibility. This 
attarctive interaction leads to fermion pairing, an effect 
that has been studied in some detail for the case that the 



bosons are deep in the superfluid phase 1^ 13, [Hi [l^l ; in 
this case the effective interaction is dominated by single 
phonon exchange, and takes a form very similar to the 
electron-phonon interaction in metals. When the bosons 
are in the Mott insulator phase x(q, w) is zero at T = 
and small w, and takes an activated behavior at low T due 
to the Mott-Hubbard gap; as a result the effective pairing 
interaction is extremely weak. In the following we study 
the pairing physics near the particle-hole asymmetric SIT 
with z = 2, where the critical properties are known and 
the Bose-Fermi interaction can be treated perturbatively 
due to its irrelevance in the RG sense. We will comment 
on the case of particle-hole symmetric SIT later on. 

The critical behavior of particle-hole asymmetric SIT 
is that of the dilute Bose gas transition 2J] . For d = 2 or 
3, the theory is at or above its upper critical dimension, 
and a mean-field (or Hartree-Fock) treatment of the bo- 
son interaction U in the calculation of x(q, w) in Eq. ([7]) 
is justifiable even near the transition. For simplicity we 
focus on c? = 3 below to avoid complications associated 
with logarithmic correction at the upper critical dimen- 
sion d = 2. For weak pairing (which is assumed here), 
we may neglect the frequency dependence in Eq. ([7]) and 
reduce it to a static attractive interaction: 



K//(q) 



(8) 



where k 



driE 



gjj^ is the boson static compressibility, and 
is proportional to the boson superfluid correlation length 
(with an 0(1) prefactor). This attractive interaction 
leads to fermion superfluid transition temperature in an- 
gular momentum channel I: 



T[ = AiEFexp{-l/[D{EF)T,]}, 



(9) 



where Ep is Fermi energy, D(Ep) is fermion density of 
states at Ep, Ai is a constant of order 1 (e.g., Ai^q « 
0.61), and 

21 + 1 f^'" 

F/ = — — / dcos6IV;//[2/cFsin(6l/2)]P/(cos6') (10) 
2 Jo 

is the pairing interaction in angular momentum channel 
Here kp is the Fermi wave vector and Pi is the Legendre 
polynomial. 

On the superfluid side with A/i > (A/i is the differ- 
ence between /xb and the phase boundary) and below the 
boson superfluid transition temperature , straightfor- 
ward calculation shows 12311 



K « 1/U 

which is mostly independent of A/i, and 



(11) 



(12) 



which diverges with mean- field exponent as Afi — > 0. The 
increasing correlation length ^ reduces the s-wave pairing 
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strength Tq. However it enhances the p-wave pairing 21 
strength Fi due to the stronger momentum dependence 
of the pairing interaction, as long as ^ < l/kp. Since for 
the single component fermion in our model ^ s-wave 
pairing is not allowed and the leading pairing instability 
is in the p-wave channel, we expect fermion gap and Tf 
to increase as one approaches SIT; thus one expects Tf 
to peak very close to the critical point with A/x = 0. 

Since ^ as A/i ^ 0, Tf wiU cross when one 
gets sufficiently close to the SIT. In this case the pairing 
interaction is controlled by the finite T properties of the 
z — 2 QCP and becomes strongly T-dependent. Again 
using mean-field approximation we obtain 



and [24 1 



K - 1/{UHbT)^^^ 



(13) 



(14) 



In this case needs to be determined self-consistently 
by solving Eq. ([9]). Our qualitative results on fermion 
pairing are summarized in Fig. 4.(25j| 

As discussed earlier, Bose-Fermi interaction is relevant 
at the particle-hole symmetric QCP of SIT, and may flow 
to a strong coupling fixed point. If this turns out to be 
the case then fermions have a universal strong pairing 
interaction at this QCP, a situation very similar to Fes- 
hbach resonance. We thus expect Tf ~ Ep, where Ep 
is the energy scale at which the Bose-Fermi interaction 
has flown to the universal value. We expect Ep ~ Ep for 
V ^ Ep, as there is no other energy scale for the fermions 
in this case. This could be an alternative way to achieve 
"high" p-wave . In both the speculative case here and 
the concrete case of particle-hole asymmetric QCP with 
z = 2 discussed above, we find p-wave Tf peaks at the 
QCP due to pairing interaction mediated by quantum 
critical fiuctuations. This is similar to the quantum crit- 
icality scenario of high Tc superconductivity in cuprates, 
and provides a concrete example of this scenario. 




FIG. 4: Finite temperature phase diagram near the particle- 
hole asymmetric boson superfluid-insulator transition, at 
Afi = and T = 0. The solid line is p-wave fermion su- 
perfluid transition temperature, while dashed line is boson 
superfluid transition temperature. 



with each other. More specifically, we have studied 
the effect of fermions on the critical properties of boson 
superfiuid-insulator transition (SIT), and the pairing in- 
teraction between fermions mediated by quantum critical 
fluctuations near SIT. In the latter case we predict the 
fermion pairing gap and superfluid transition tempera- 
ture peak near boson SIT. It would be very interesting 
to use quantum Monte Carlo or other numerical methods 
to test the predictions of this paper, especially to resolve 
the nature of the particle-hole symmetric boson SIT in 
the presence of fermions. 
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CONCLUDING REMARKS 

Bose-Fermi mixtures represent new quantum many- 
body systems realized in trapped cold atomic gases, that 
do not have straightforward correspondences in electronic 
condensed matter systems. In the latter case the con- 
stituent particles, namely electrons, are fermions. When 
the electrons form Cooper pairs, the system may also be 
described (at low energy) using boson only models like 
the Bose Hubbard model; in these cases typically the 
fermionic degrees of freedom are fully gapped and do not 
enter the low-energy effective theory. 

In the present paper we have explored some new 
physics in Bose-Fermi mixtures, in which low-energy 
bosonic and fermionic excitations co-exist and interact 
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